The Computational Notebook Paradigm for Multi-Paradigm Modeling

Bentley James Oakes, Romain Franceschini, Simon Van Mierlo, Hans Vangheluwe

Introduction

  • Motivation: Reproducibility in Science
  • Computational Notebook (CN) Example
  • CN Paradigm
  • CN Paradigm for Multi-Paradigm Modeling

Reproducibility

Repeatability = same team and same methods -> same results (with some measure of precision).

Replicability = different team and same methods -> same results.

Reproducibility = different team and different methods -> same results.

S. Schnell, ““Reproducible” Research in Mathematical Sciences Requires Changes in our Peer Review Culture and Modernization of our Current Publication Approach,” Bulletin of Mathematical Biology, vol. 80, no. 12, pp. 3095–3105, Sep. 2018.

Reproducibility figure from: Reproducible Research in Data Science https://github.com/mdozmorov/presentations/

For experimental reproduction, need to document and disseminate:

  • Scientific notes, such as:
    • Experimental conditions, data
    • Processes, observations, and insights
  • In computational domain, further barriers to reproducibility:
    • Hardware and software versioning
    • Program state
      • Ex. seed for random number generation
  • Multi-paradigm modeling can assist reproducibility by providing explicit, standardized syntax and semantics. An example: Petri net formalism

How can computational notebooks assist?

Computational Notebooks

  • Similar to the read-evaluate-print-loop (REPL) of a shell
  • Execute code, examine the result
  • Notebooks also add:
    • Record of code executed
    • Documentation (text, images, video, website embeds, etc.)

Code Example:

In [1]:
import datetime
x = datetime.datetime.now()
print("Hello MPM4CPS! The time is: " + str(x))

Hello MPM4CPS! The time is: 2019-09-17 14:06:56.311601
  • Notebook entries are Input cells and Output cells
    • Input cells are executed with a particular Kernel
    • Example: Python code is executed with the IPython kernel
    • Result visualized in paired output cell
  • Input cells can be repeatedly modified and re-executed

Bouncing Ball Example

A small example of a bouncing ball model follows, implemented as a black-box in the Functional Mock-up Interface (FMI) standard.

Objective: Show experimentation and documentation processes within the CNP.

The equations and initial conditions of the model:

In [2]:
%%latex
\begin{equation}
\frac{dh}{dt} = v; \frac{dv}{dt} = -g;\\
\mathit{when}~h < 0~\mathit{then}~v := -e * v;\\
\mathit{initial:}~e = 0.7, g = 9.81
\end{equation}
\begin{equation} \frac{dh}{dt} = v; \frac{dv}{dt} = -g;\\ \mathit{when}~h < 0~\mathit{then}~v := -e * v;\\ \mathit{initial:}~e = 0.7, g = 9.81 \end{equation}
In [3]:
from pyfmi import load_fmu
model = load_fmu('bouncingBall.fmu')
In [4]:
import pylab as P

def plot(res):
    fig = P.figure(figsize=(3, 1.75),)
    P.clf()
    P.plot(res['time'], res['h'])
    P.ylabel('Height (m)')
    P.xlabel('Time (s)')
    P.suptitle('FMI Bouncing Ball')
    P.show()
In [5]:
model.reset()
model.set('h',200)
result = model.simulate(final_time = 20.)
plot(result)

Can experiment with h as 100, 300, 0.01, 0, and observe results

Computational Notebook Paradigm

  • Entities and Processes and their relationship to Multi-Paradigm Modeling (MPM)
    • MPM : explicit modeling of every (relevant) aspect of a system
      • most appropriate level(s) of abstraction
      • most appropriate formalism(s), and
      • modeling of the process(es)
  • Compatibility points between the CNP and MPM

Notebook Entities

Let's divide entities into three sections:

  • Input and Output cells
  • Languages and Kernels
  • Users and Processes

Notebook Entities - Cells

The notebook is a linear presentation of cells.

Notebook Entities - Languages

  • Languages/Kernels/Visualizer entities represent different formalisms

Notebook Entities - Users/Processes

Users interact with notebooks in various processes, and produce artefacts (such as PDFs or LaTeX)

Process - Experimentation

  • Umbrella term: Optimization, verification, simulation,
  • Objective is exploring the system under study
    • Reproducibility: Allow same/different team to understand system behaviour
  • Computational notebook can be enriched with MPM for full-system evaluation
    • Capture properties of interest, design, property-checking processes together
    • Provide different views on the system in different cells or notebooks
      • Example: Bouncing ball example in the Causal-Block Diagram formalism

Bouncing Ball Example in Causal Block Diagram Formalism

In [6]:
import pydot
from cbd.BouncingBallCBD import BouncingBallCBD as bbCBD
cbd = bbCBD("myCBD")
cbd.draw()
In [7]:
cbd.simulate()
cbd.draw_scopes()
  • Showing views allow teaching of how views/formalisms impact system properties
    • Example: Do the FMI model and the CBD model produce equivalent results?
  • Open question: How to structure these different views into a workflow/process?
    • Linear presentation of notebooks may make this impractical
    • Explicitly model the workflow using MPM techniques

Process - Documentation

  • Notebooks retain code and results for future access
  • Can produce artefacts (PDF, LaTeX, online notebook)
    • Section in paper produced from notebook LaTeX
  • Notebooks can be combined with other technology for greater reproducibility
  • Binder (mybinder.org) hosts online notebooks
    • Hosts a container with software dependencies
    • Bouncing ball notebook and this presentation available at: 
      * https://mybinder.org/v2/git/http%3A%2F%2Fmsdl.uantwerpen.be%2Fgit%2Fbentley%2FNotebookParadigmPaper2019-presentation.git/master

Computational Notebook Paradigm (CNP) for Multi-Paradigm Modeling (MPM)

High degree of compatibility between the CNP and MPM:

  • Kernels <-> Formalisms/Abstractions
  • User/Notebooks <-> Processes

Next slides will discuss three areas where CNP could better support MPM:

  • Model Frames
  • Domain-Specific Languages (DSLs)
  • History and Traceability

Model Frames

Has three components:

a. a modeling activity such as calibration or verification, with inputs, outputs, and a process description

b. a context with objectives, assumptions, and constraints, and

c. zero or more sub-frames.

Example for the bouncing ball model:

Model Frame Example

In [8]:
# check if parameters above 1m height, between -65 and 180 C
# assume that all results of the simulation are valid
def bb_frame(height, temp, t):
    if height < 1:
        raise Exception("Invalid height!")
    if temp < -65 or temp > 180:
        raise Exception("Invalid temperature!")
        
    model.reset()
    model.set('h',height)
    return model.simulate(final_time = t)
       
result = bb_frame(height=100, temp=25, t=20)
plot(result)
  • Validates parameter values and simulator output
  • Further steps: Precise simulator initialization, temporal logic checking, configuration frames

Domain-Specific Languages

User may want to create languages to specify models/experiments during the computational notebook activity

BouncingBallLang Example:

In [9]:
#BouncingBallLang syntax
#[BALL_ID HEIGHT]
In [10]:
#BouncingBallLang semantics
def bb_execute(statement):
    ball_id = statement[0]
    height = statement[1]
    model.reset()
    model.set('h',height)
    print("Result for bouncing ball '" + ball_id +"':")
    plot(model.simulate(final_time = 20))
In [11]:
#BouncingBallLang Usage
ball = ["BALL1", 160]
bb_execute(ball)

Result for bouncing ball 'BALL1':

Updating language leads to inconsistencies. Can these be detected/fixed automatically?

History and Traceability

  • In (most) notebooks, history of cells is not retained.
    • Could be useful for understanding notebook/system evolution
  • There is also no mechanism for representing relationships between cells
    • Such as conformance between the bouncing ball language and its usage
    • Could enable syntax checking, model evolution

Conclusion and Future Work

  • Demonstrated the computational notebook paradigm (CNP) and benefits:
    • Enhance reproducibility
    • Communicate results and understandings
  • Discussed interactions between CNP and multi-paradigm modelling (MPM)
    • Natural integration of CNP kernels and MPM techniques
    • Highlighted areas of improvement
  • Future Work
    • Implement MPM-based kernels, such as connection to the ModelVerse
    • Examine model frames and their integration
    • Represent workflow processes explicitly Ex. steps for creating DSL

Presentation available as an online Jupyter notebook: https://mybinder.org/v2/git/http%3A%2F%2Fmsdl.uantwerpen.be%2Fgit%2Fbentley%2FNotebookParadigmPaper2019-presentation.git/master (or http://bit.do/e6yL8)